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1.  Introduction 


In  2004,  the  U.S.  Army  Tank- Automotive  Research  Development  and  Engineering  Center 
(TARDEC),  in  partnership  with  the  U.S.  Army  Research  Laboratory  (ARL)  Human  Research 
and  Engineering  Directorate  (HRED),  pursued  a  5-year  program  U.S.  Army  Technology 
Objective  (ATO).  The  purpose  of  the  ATO  was  to  develop  the  tools,  techniques,  and  autonomy 
to  maximize  mounted  and  dismounted  control  of  ground  and  air  unmanned  systems  and  optimize 
Soldier-robot  and  robot-robot  ground  and  air  teams.  Development  included  a  scalable  user 
interface  for  robotic  control.  The  interface  maximizes  multi-function  Soldier  perfonnance  for 
primary  tasks  while  minimizing  unique  training  requirements,  achieved  by  optimizing  and 
standardizing  the  required  interactions  and  managing  the  workload  associated  with  the  control  of 
unmanned  ground  and  air  systems.  This  report  highlights  the  Robotics  Collaboration  ATO 
Capstone  Experiment  on  small  robot  control. 

1.1  General  Background 

As  the  emerging  technologies  of  the  Army’s  Future  Combat  System  (FCS)  are  introduced  to  the 
battlefield,  Soldiers  will  increasingly  face  new  challenges  in  workload  management.  A  shifting 
force  structure  will  bring  increasing  responsibilities  for  the  next  generation  Soldier,  who  will  be 
tasked  with  effectively  using  and  protecting  robotic  assets  in  addition  to  performing  other 
primary  missions.  TARDEC  and  HRED  are  addressing  issues  in  human-robot  interaction  (HRI) 
under  the  Robotics  Collaboration  (RC)  ATO  program.  The  key  program  goal  is  to  understand 
HRI  issues  in  order  to  develop  technologies  and  mitigations  that  enhance  HRI  perfonnance  in 
future  combat  environments. 

Robots  have  been  and  continue  to  be  effective  in  combat  tasks  that  are  “dull,  dirty,  and 
dangerous”  (Carafano  and  Gudgel,  2007).  Remotely  controlled  robotic  unmanned  air  vehicles 
(UAVs)  and  unmanned  ground  vehicles  (UGVs)  play  a  vital  role  in  current  operations,  including 
disarming  roadside  bombs  and  perfonning  ground-  and  air-based  patrol  missions.  However, 
these  systems  involve  the  cognitive  resources  of  the  operators,  who  are  continuously  engaged  in 
teleoperating  the  systems  and  manipulating  the  sensors  or  payloads  during  the  mission.  Future 
military  operations  require  the  Soldier  to  be  as  unencumbered  by  equipment  and  weapon 
constraints  as  possible.  At  the  same  time,  the  number  of  possible  systems  that  are  available  at 
even  the  squad  level  will  increase  dramatically  from  the  current  force.  In  particular,  the  use  of 
robotic  systems  will  enhance  the  operator’s  safety,  flexibility,  and  lethality  during  urban 
missions — but  at  a  price.  Performance  is  enhanced  by  the  ability  to  remotely  “see”  and  “shoot” 
in  buildings,  disann  explosive  devices,  and  view  around  corners.  The  price  is  the  additional 
workload  and  loss  of  situation  awareness  (SA)  when  the  Soldier  is  most  vulnerable.  Robotic 
teleoperation  focuses  the  operator  on  the  robotic  task,  to  the  detriment  of  personal  security  (Chen 
and  Joyner,  2009). 
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The  twin  goals  of  the  Soldier-robot  teaming  research  were  to  understand  HRIs  in  a  military 
environment  and  assess  technologies  that  mitigate  undesirable  effects.  One  area  of  interest  was 
the  use  of  innovative  solutions  such  as  multimodal  interfaces.  In  general,  ARL  researchers  found 
that  using  tactile  and  auditory  cueing  both  separately  and  as  redundant  cues  improved  Soldier 
multitasking  performance  (Chen,  Haas,  and  Bames,  2007;  Chen  and  Terrence,  2008;  Haas, 

2007).  Tactile  cueing  was  found  to  be  more  flexible  because  it  did  not  require  the  Soldier  to 
disambiguate  between  front  and  back  signals  for  navigation  tasks.  Perhaps  more  importantly, 
tactile  cueing  uses  communications  channels  that  are  currently  underused  in  military 
environments  (Barnes,  in  press).  For  example,  our  research  verified  that  tactile  information 
could  be  used  for  covert  communications  using  a  limited  lexicon  that  was  easily  learned  by 
Soldier  participants  (Gilson,  Redden,  and  Elliott,  2007).  Ease  of  learning  tactile  cues  was  a 
reoccurring  trend  in  many  of  the  published  studies,  further  suggesting  the  advantages  of  tactile 
communications  for  Soldier  robot  teaming  operations  (Jones  and  Sarter,  2008). 

For  example,  Pettit,  Redden,  and  Carstens  (2006)  translated  Army  hand  and  arm  signals  to  a 
covert  tactile  communication  system,  resulting  both  in  improved  navigation  aids  and  as  a  silent 
signaling  environment  for  army  snipers  (Gilson  et  ah,  2007).  However,  the  potential  uses  of 
covert,  low-bandwidth  tactile  modes  go  beyond  communication  networks  among  Soldiers.  As 
robots  develop  more  environmental  awareness,  tactile  communications  might  be  possible  as  a 
two-way  conduit  between  robots  and  Soldier  teams  (Barnes,  in  press).  The  current  study  was 
designed  to  explore  this  possibility  in  a  realistic  field  environment  at  Ft.  Bliss  in  El  Paso,  TX, 
with  experienced  Soldiers.  Soldiers  conducted  search  missions  in  desert  environment,  not  too 
unlike  the  Iraqi  desert.  The  Soldier  operators  were  given  minimal  training  using  a  TALON  robot 
with  three  communication  modes:  chat  (visual  text),  radio,  and  tactile.  The  Soldier  operator  was 
directed  by  a  remotely  located  commander  to  target  locations  hidden  in  the  desert  terrain.  The 
operator  in  turn  directed  the  TALON  robot  to  various  locations  using  a  specially  designed 
interface  based  on  the  commander’s  inputs  in  one  of  the  three  communications  modes.  This 
emulated  Soldier  teams  or  Soldier  robot  teams  working  in  close  concert  and  sharing  SA. 

1.2  Soldier  Robot  Teaming  Objective 

The  objective  of  the  current  research  was  to  explore  potential  relationships  between  modes  of 
communication,  i.e.,  auditory,  visual,  or  tactile,  and  task  performance  for  shared  control  of 
dismounted  operations  with  a  small  unmanned  ground  vehicle  (SUGV).  Specifically,  the 
relationships  were  explored  in  a  SUGV  teleoperation  task  in  which  critical  navigation  and  map 
information  was  the  focus  of  the  dismount  team’s  shared  control. 

1.3  Research  Questions 

This  research  sought  to  address  the  impact  of  the  type  of  communication,  i.e.,  auditory,  visual,  or 
tactile,  on  a  dismounted  team’s  performance  on  a  target  identification  task.  To  help  answer  the 
overall  question,  several  specific  questions  were  addressed  in  the  experiment. 
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1.3.1  Time  to  Target 

Are  there  differences  between  the  three  levels  of  communications  type  (radio,  chat,  and  tactile) 
with  regard  to  the  average  time  to  identify  a  target?  Are  there  differences  between  the  three 
levels  of  target  type  (large,  medium,  and  small)  with  regard  to  the  average  time  to  identify  a 
target?  Is  there  an  interaction  between  communications  type  and  target  type  with  regard  to  the 
average  time  to  identify  a  target? 

1.3.2  Number  of  Communications 

Are  there  differences  between  the  three  levels  of  communications  type  (radio,  chat,  and  tactile) 
with  regard  to  the  average  number  of  communications  required  to  identify  a  target?  Are  there 
differences  between  the  three  levels  of  target  type  (large,  medium,  and  small)  with  regard  to  the 
average  number  of  communications  required  to  identify  a  target?  Is  there  an  interaction  between 
communications  type  and  target  type  with  regard  to  the  average  number  of  communications 
required  to  identify  a  target? 

We  hypothesized  that  the  tactile  communications  would  result  in  improved  latency  and  accuracy 
performance  compared  to  traditional  communication  modes  (i.e.,  radio). 


2.  Method 


2.1  Participants 

Volunteer  participants  consisted  of  13  active-duty  Soldiers  from  the  2nd  Combined  Arms 
Battalion  5th  Brigade  (Army  Experiment  Task  Force)  located  at  Ft.  Bliss,  TX.  All  of  the 
Soldiers  were  male  ranging  in  age  from  20-3 1  with  an  average  of  24.08  years  of  age. 

2.2  Apparatus 
2.2.1  TALON  Robots 

The  SUGV  for  this  experiment  was  the  TALON  robot.  The  TALON  is  a  man-portable  (45  kg) 
rugged  robot  that  can  negotiate  through  all  types  of  terrain  and  can  withstand  all  types  of  weather 
(figure  1). 
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Figure  1.  TALON  robot. 

2.2.2  AMREL  Dismounted  Controller 

The  controller  of  the  SUGV  allows  the  dismounted  Soldier  to  operate  the  SUGV  and  share 
information  with  his  team  member  about  mission-critical  events.  It  is  a  tablet-based  controller 
(figure  2)  with  a  resistive  touch  screen  and  a  stylus  that  is  ruggedized  for  the  extreme  weather 
conditions  and  suited  with  military  connection  capability.  The  controller  has  the  following 
dimensions: 

•  9.8  in  wide  x7.4  in  diameter  x  1.65  in  high 

•  4.2  lb  with  primary  battery 

•  4.7  lb  with  second  battery 


Figure  2.  AMREL  dismounted  controller. 
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2.2.3  Tactor  Belt 


The  tactor  belt  was  worn  around  the  waist  and  contained  eight  individual  tactors  (the  C-2  model, 
Engineering  Acoustics,  Inc.;  figure  3).  The  group  of  tactors  functioned  together  and  vibrated  in  a 
unique  pattern  to  distinguish  discrete  commands. 


F igure  3 .  T actor  belt. 


2.2.4  Targets 

Large,  medium,  and  small  targets  were  used  in  each  experimental  mission.  A  mission  was  the 
experimental  course.  The  large  target  was  a  construction  barrel  ~4  ft  high.  The  medium  target 
was  an  orange  cone  ~18  in  high.  The  small  target  was  a  white  pipe  ~12  in  long. 

2.2.5  Questionnaires 

Subjective  workload  and  SA  ratings  were  measured  with  the  National  Aeronautics  and  Space 
Administration-Task  Load  Index  (NASA-TLX)  (Hart  and  Staveland,  1987)  and  the  Cognitive 
Compatibility  Situation  Awareness  Technique  Questionnaire  (CC-SART)  (Taylor,  1990), 
respectively.  The  NASA-TLX  is  a  multi-dimensional  rating  procedure  that  derives  an  overall 
workload  score  based  on  a  weighted  average  of  ratings  on  six  subscales  (Mental  Demand, 
Physical  Demand,  Temporal  Demand,  Own  Performance,  Effort,  and  frustration).  The  CC- 
SART  is  a  subjective  rating  of  SA,  in  which  participants  rate  their  experience  with  the  task  on 
three  dimensions  (Activation  of  Knowledge,  Ease  of  Reasoning,  and  Level  of  Processing).  Level 
of  Processing  is  the  degree  to  which  the  situation  involves,  at  the  low  level,  natural  automatic, 
intuitive,  and  associated  processing  or,  at  the  high  level,  analytic,  considered,  conceptual,  and 
abstract  processing.  Ease  of  Reasoning  is  the  degree  to  which  the  situation,  at  a  low  level,  is 
confusing  and  contradictory  or,  at  a  high  level,  is  straightforward  and  understandable.  Activation 
of  Knowledge  is  the  degree  to  which  the  situation,  at  the  low  level,  is  strange  and  unusual  or,  at 
the  high  level,  is  recognizable  and  familiar.  Workload  and  SA  were  assessed  at  the  end  of  each 
mission. 
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2.3  Experiment  Design  and  Methodology 
2.3.1  Experimental  Design 

The  experiment  was  a  3  x  3  within-subject  design.  Communication  type  and  target  type  were 
two  independent  variables  of  experimental  interest.  Communication  type  had  three  categories: 
radio  (audio),  chat  (visual),  and  tactile.  Target  type  had  three  levels  (large,  medium,  and  small). 
Participants  completed  two  missions  in  each  of  the  three  communication  conditions;  therefore, 
each  participant  completed  six  missions.  A  mission  consisted  of  the  participant  driving  the  robot 
through  a  course  with  embedded  targets  (one  large  target,  one  medium  target,  and  two  small 
targets)  that  he  had  to  identify. 

There  were  three  courses  used  in  the  experiment.  Due  to  logistical  constraints,  the  three  courses 
were  set  up  prior  to  the  experiment  and  target  locations  were  fixed  for  the  duration  of  the 
experiment.  The  size  of  the  course  areas,  the  terrain  features,  and  the  distances  between  targets 
within  each  course  were  approximately  equivalent  between  the  courses.  On  each  course  there 
were  four  targets  randomly  dispersed  in  the  course  area.  So  in  one  course  the  first  target  could 
be  a  small  target  and  in  another  course  the  first  target  could  be  a  large  target.  Participants  ran  the 
robot  through  each  course  in  succession,  course  1,  course  2,  and  then  course  3.  This  sequence 
was  repeated  for  a  total  of  six  missions.  Table  1  shows  the  randomly  assigned  order  of  exposure 
to  the  various  communication  conditions  and  course  layouts. 

Table  1 .  Randomly  assigned  exposure  orders  of  communication  conditions  and  course  layouts. 


Participants 

Mission  1 

Mission  2 

Mission  3 

Mission  4 

Mission  5 

Mission  6 

Course  1 

Course  2 

Course  3 

Course  1 

Course  2 

Course  3 

1,  12,  13 

Radio 

Radio 

Chat 

Chat 

Tactor 

Tactor 

2,11 

Tactor 

Tactor 

Chat 

Chat 

Radio 

Radio 

3,  10 

Radio 

Radio 

Tactor 

Tactor 

Chat 

Chat 

4,9 

Tactor 

Tactor 

Radio 

Radio 

Chat 

Chat 

5,  8 

Chat 

Chat 

Radio 

Radio 

Tactor 

Tactor 

6,7 

Chat 

Chat 

Tactor 

Tactor 

Radio 

Radio 

The  three  types  of  communication  conditions  produced  a  total  of  six  possible  exposure  orders. 
Repetitions  of  the  six  potential  exposure  orders  to  the  three  communications  conditions  was 
balanced  to  the  best  extent  possible  across  13  participants  such  that  two  of  the  exposure  orders 
were  assigned  two  repetitions  and  one  was  allowed  3  repetitions.  The  assignment  of  participant 
to  exposure  condition  was  random  without  replacement.  Radio,  then  chat,  and  then  tactors  was 
the  exposure  order  with  three  randomly  assigned  participants. 

2.3.2  Experimental  Dismount  Team 

The  experimental  team  consisted  of  a  commander,  a  robotic  controller,  and  a  robotic  follower. 
The  role  of  the  commander  was  filled  by  an  experimental  team  member.  The  commander  was  in 
charge  of  relaying  navigational  commands  to  the  robotic  controller,  directing  him  towards  the 
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targets.  The  role  of  the  robotic  controller  was  filled  by  a  Soldier  participant.  The  robotic 
controller  was  in  control  of  the  TALON  mobility  and  responsible  for  identified  targets.  The  role 
of  the  robotic  follower  was  filled  by  an  experimental  team  member.  The  robotic  follower  was 
responsible  for  the  safety  of  the  TALON  while  on  the  course. 

2.3.3  Communication  Type 

Three  communication  modalities  were  used  in  the  study:  Chat  interface,  tactor  belt,  and  radio. 
The  chat,  or  visual,  modality  provided  the  participant  (i.e.,  robotic  controller)  and  the 
commander  with  a  suite  of  communications  on  the  AMREL  tablet.  A  discrete  set  of 
communications  were  represented  with  interface  buttons,  one  button  per  communication. 
Selecting  one  of  the  buttons  in  the  suite  sent  the  representative  chat  message  to  the  team  member 
and  was  displayed  as  a  text  message  on  the  AMREL  tablet.  The  tactor  belt  contained  eight 
individual  tactors.  The  tactors  functioned  together  and  vibrated  in  a  unique  pattern  for  each  of 
the  discrete  communications.  As  in  the  chat  condition,  selecting  one  the  buttons  in  the  suite  of 
discrete  communications  on  the  AMREL  tablet  sent  the  representative  message  to  the  team 
member;  however,  in  this  condition,  the  team  member’s  belt  would  vibrate  according  to  the  type 
of  message  sent.  The  radio  condition,  or  auditory  modality,  used  hand-held  two-way  radios.  All 
communications  were  shared  using  the  radios,  but  the  types  of  communications  were  restricted  to 
the  same  set  of  discrete  communications  that  were  available  in  the  chat  and  tactor  conditions. 

The  commander  and  the  robotic  operator  sent  and  received  communications  via  the  AMREL 
displays  by  using  either  tactile  belt  (wore  by  commander  and  operator)  or  chat  messaging. 

Communication  between  the  commander  and  the  participant  in  the  dismount  team  was  two  way 
within  the  communication  modality  of  the  condition.  The  commander  in  the  dismount  team  had 
a  digital  map  on  his  AMREL  display  showing  the  current  SUGV  position  (from  GPS)  and 
orientation  as  well  as  target  locations.  The  commander  directed  the  participant  with  navigational 
commands.  The  participant  teleoperated  the  TALON  to  the  critical  areas  based  on  the 
commander’s  direction;  however,  the  commander  did  not  “micromanage”  the  operator  (i.e., 
specify  which  way  to  go  around  bushes).  Using  the  condition- appropriate  communication 
modality,  the  commander  indicated  when  the  robot  was  in  the  target  vicinity  and  the  participant 
would  scan  to  find  the  target.  Commands  were  limited  to  a  discrete  set  of  1 1  communications: 

•  Scan  forward 

•  Scan  left 

•  Scan  right 

•  Transit  180° 

•  Transit  forward 

•  Transit  left 
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•  Transit  right 

•  Found 

•  Halt 

•  Rally 

•  Move  out 

2.3.4  Dependent  Variables 

Two  primary  dependent  measures  were  collected:  time  to  identify  targets  and  the  total  number  of 
navigational  communications. 

•  Time  to  identify  targets  was  defined  as  the  number  of  seconds  from  when  the 
communication  to  “move  out”  was  given  at  the  beginning  of  a  course  segment  until  the 
“found”  communication  was  given  upon  identifying  a  target. 

•  Number  of  navigational  communications  was  defined  as  the  frequency  of  navigation 
communications  issued  from  when  the  communication  to  “move  out”  was  given  at  the 
beginning  of  a  course  segment  until  the  “found”  communication  was  given  upon 
identifying  a  target. 

2.3.5  Procedure 

The  participant  was  verbally  briefed  on  his  role  in  the  teleoperation  tasks  in  the  experiment.  The 
brief  included  a  description  of  the  modes  of  communication  and  target  types  central  to  the 
experimental  tasks,  as  well  as  a  task  demonstration.  The  Soldier  then  practiced  the  experimental 
tasks  with  each  communication  modality  on  a  separate  training  course  with  targets.  Practice  was 
structured  such  that  a  communication  modality  was  first  demonstrated  to  the  Soldier,  and  then 
the  Soldier  completed  a  training  run  within  that  communication  modality,  repeating  this  practice 
structure  across  all  three  communication  conditions.  Radio  was  always  introduced  first,  then 
chat,  and  lastly  tactor.  The  exception  to  this  practice  procedure  was  for  the  tactor 
communication  modality.  The  Soldier  experienced  every  communication  condition  multiple 
times,  and  then  recognition  of  communications  was  tested  until  100%  accuracy  for  identifying 
communications  was  achieved.  This  recognition  test  was  repeated  later,  just  prior  to  beginning 
the  tactor  experiment  trials  for  further  practice  and  competence. 

The  Soldier  then  began  the  sequence  of  experimental  missions.  Each  participant  completed  one 
mission  in  a  single  modality,  and  then  repeated  the  mission  in  the  same  modality  a  second  time. 
The  two  repetitions  of  each  of  the  three  modalities  yielded  a  total  of  six  missions  per  participant. 
Immediately  following  the  first  pair  of  sequential  missions  within  a  single  communications 
modality,  the  Soldier  was  administered  the  NASA-TLX  and  SART  questionnaires.  The  Soldier 
performed  the  second  pair  of  missions  followed  with  associated  questionnaires;  and  lastly  the 
third  pair  of  missions  with  associated  questionnaires.  Within  a  mission,  the  procedure  was  to 
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first  get  verbal  confirmation  from  the  Soldier  of  readiness  to  begin.  The  commander  then  issued 
a  “move  out”  command  to  begin  the  mission,  followed  by  a  series  of  additional  navigational 
commands  directing  the  Soldier  to  each  target  within  the  mission.  The  participant  reported 
“Target  found”  upon  identifying  each  target  and  the  commander  would  respond  with  a 
“Confirm.”  After  receiving  the  confirmation,  the  Soldier  progressed  to  the  next  target  in  the 
mission.  Upon  identifying  the  fourth  target  in  the  mission,  the  commander  would  issue  a  “Rally” 
command  signifying  the  end  of  the  experimental  trial  and  indicating  that  the  Soldier  was  to 
teleoperate  the  TALON  back  to  the  mission  start  point. 

2.4  Data  Collection 

2.4.1  Data  Processing 

The  data  for  each  participant  were  captured  in  the  AMREL  event  data  logs  and  by  an 
experimental  observer;  however,  the  event  data  logs  were  incomplete,  including  complete  sets  of 
only  chat  and  tactor  data.  Radio  communications  via  the  two-way  radio  was  independent  of  the 
data  logging  mechanism.  The  event  data  logs  recorded  with  millisecond  precision  represented  a 
potentially  more  accurate  source  of  target  times  than  the  records  of  the  experimental  observer, 
but  the  records  kept  by  the  experimental  observer  were  complete  for  all  three  conditions,  even 
though  no  doubt  subject  to  rounding  bias  due  to  the  hand  recorded  event  records.  In  order  to 
ensure  accurate  target  time  data,  in  the  conditions  where  the  redundant  data  existed,  the  time  to 
each  target  within  each  communication  condition  was  examined  for  both  the  data  logs  and  the 
observer  records.  The  examination  revealed  near  equivalent  records  with  slightly  shorter 
observer  record  times  than  data  log  times.  Since  the  observer  records  were  complete,  consistent 
across  all  conditions,  and  nearly  equivalent  to  the  data  logs  in  redundant  conditions,  the  observer 
records  were  selected  for  the  time  analysis.  Otherwise,  the  three  modes  would  be  compared 
using  two  different  recording  mechanisms:  the  AMREL  for  the  chat  and  tactor  data  and  the 
manual  recording  for  the  audio  data,  leaving  open  the  possibility  of  a  confound  between  type  of 
recording  and  communication  mode.  In  the  observer  records,  there  were  16  total  instances  of 
either  missing  or  ambiguous  mission  data.  Each  event  was  isolated  and  triangulated  with  data 
log  elapsed  times,  observer  record  elapsed  times,  and  observer  trial  notes.  Each  of  the  16 
missing  or  ambiguous  target  times  was  corrected  using  time  estimates  across  the  three  sources. 

To  ensure  the  communication  frequency  data  was  accurate,  the  number  of  communications  per 
target  within  each  communication  condition  was  examined  in  both  the  data  logs  and  the  observer 
records,  again  in  the  conditions  where  the  redundant  data  existed.  The  examination  of 
communication  frequencies  revealed  that  records  between  the  two  sources  closely  matched  in  the 
chat  and  tactor  conditions,  but  varied  considerably  in  the  radio  condition.  The  only  means  of 
communicating  in  the  chat  and  tactor  conditions  was  via  the  tablet,  and  therefore,  transmitted 
with  the  tablet,  and  as  a  result,  captured  in  the  logs.  Consequently,  the  data  logs  had  to  be  100% 
accurate  for  these  two  conditions  and  were  preserved  for  analysis.  This  was  not  the  case  for  the 
radio  condition.  Further  examination  of  the  observer  trial  notes  and  observer  records  revealed 
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that  repeated  radio  commands  were  recorded  in  the  observer  records,  but  only  a  single  instance 
of  the  corresponding  command  was  recorded  in  the  data  logs.  It  was  inferred  that  in  instances 
when  the  verbal  radio  command  had  to  be  repeated  several  times  by  the  commander,  the 
commander  only  entered  the  command  in  the  tablet  once;  perhaps  due  to  the  multitasking 
demand  of  using  the  radios  and  tablet  simultaneously.  This  provided  evidence  that  the  data  logs 
were  likely  less  accurate  in  radio  command  frequency  than  the  observer  records  and  that  the 
observer  records  for  radio  command  frequency  were  a  valid,  reliable,  and  unbiased  substitution 
for  analysis. 


3.  Analyses  and  Results 


3.1  Time  to  Identify  Targets 

3.1.1  Data  Screening  and  Selection 

The  time  data  for  each  participant  was  first  plotted  by  communications  type  and  target  type,  and 
then  visually  inspected.  It  was  observed  that  the  majority  of  the  outlying  or  extreme  values 
existed  for  trials  in  excess  of  150  s.  A  review  of  the  observer  run  sheets  indicated  that  trials 
approximating  or  exceeding  150  s  often  involved  errors  of  some  kind.  Example  errors  were 
communication  failures,  failure  of  AMREL  dismounted  controller  software,  TALON 
malfunctions,  or  simply  participant  confusion.  The  data  did  not  meet  nonnality  assumptions, 
showing  evidence  of  skew,  kurtosis,  and  significant  non-normality  according  to  the  Shapiro- 
Wilk  test  of  normality.  A  log  transfonnation  reduced  skew  to  two  instances  and  kurtosis  to  three 
instances,  and  only  one  condition  failed  the  test  of  nonnality;  however,  filtering  data  points  in 
excess  of  150  s  reduced  skew  to  0  instances  and  kurtosis  to  two  instances,  and  resulted  in 
nonnally  distributed  data  in  all  conditions.  Due  to  better  meeting  the  normality  assumptions 
with  the  filtered  data,  analysis  was  carried  out  with  this  latter  data  set. 

3.1.2  Analysis 

A  3  x  3  repeated  measures  analysis  of  variance  (ANOVA)  was  performed  on  the  time-to- 
identify-targets  data  based  on  communication  type  and  target  type. 

3.1.3  Results 

The  interaction  of  communications  type  x  target  type  was  not  significant,  F(4,48)=1.01,/»<.41, 
rjp  =0.08.  Communications  type  was  not  significant,  F(2,24)=l  .3,  p<29,  yjp  =0. 1 .  Target  type 
was  significant,  suggesting  an  effect  of  the  size  of  targets  on  RT,  F(2,24)=9.72,/K.001, 

>1P  =0.45.  The  mean  time  to  identify  for  the  large  target  was  M= 66.99s,  for  the  small  target  was 
M=80.34  s,  and  the  medium  target  was  M=  87.17s.  Post-hoc  tests  using  the  Bonferroni 
adjustment  for  multiple  comparisons  revealed  that  Soldiers  were  significantly  faster  identifying 
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the  large  targets  versus  the  small  or  medium  targets,  but  that  the  results  for  the  small  target  did 
not  differ  from  the  medium  target.  The  significant  post-hoc  results  are  shown  in  table  2. 

Table  2.  Post-hoc  tests  on  time  to  identify  (seconds)  by  target  type. 


Mean  Diff 

SE 

Sig 

d' 

Large  target 

Medium  target 

13.35 

3.71 

0.01 

0.93 

Large  target 

Small  target 

20.18 

5.62 

0.01 

1.07 

3.2  Number  of  Navigational  Communications 

3.2.1  Data  Screening  and  Selection 

The  number  of  navigational  communications  for  each  participant  in  each  condition  were  plotted 
by  target  type  and  visually  inspected.  Visually  nothing  appeared  out  of  normal  range,  but  the 
data  within  conditions  did  show  evidence  of  minor  skew,  kurtosis,  and  small  departures  from 
nonnality  using  the  Shapiro-Wilk  test  of  normality.  A  natural  log  transformation  was  perfonned 
on  the  data  to  eliminate  these  problems.  The  transformation  corrected  any  normality  violations. 
Log  linear  is  similar  to  a  chi  square  in  that  it  can  handle  interactions,  which  makes  it  a 
nonparametric  equivalent  to  the  3^3  parametric  ANOVA  concurrently  performed  on  the  data. 
The  two  complimentary  analyses  provide  a  more  comprehensive  evaluation  of  the  data  so  that 
we  could  make  a  better  judgment  about  patterns  in  the  data.  Additionally,  since  the 
communication  data  were  frequency  counts,  they  were  also  compiled  as  total  conditional 
frequencies  for  a  tabular  analysis.  The  total  number  of  communications  was  counted  across  all 
participants  within  each  condition.  Due  to  two  small  targets  in  each  mission,  the  large  and 
medium  targets  each  had  two  repetitions  across  conditions,  but  the  small  target  segment  had  four 
repetitions.  To  account  for  the  extra  repetitions  for  the  small  target,  the  total  frequency  was 
calculated  and  then  divided  by  two  to  make  the  small  target  frequency  equivalent  relative  to  the 
large  and  medium  targets  frequencies. 

3.2.2  Analyses 

First,  a  3  x  3  repeated  measures  ANOVA  was  perfonned  on  the  navigational  communications 
data  with  communication  type  and  target  type.  Next,  a  log  linear  analysis  was  performed  on  the 
cross  tabulated  navigational  communication  frequency  data. 

3.2.3  Results 

The  interaction  of  communications  type  x  target  type  was  not  significant,  F(4,48)=1.23, /><.31, 
r\p  =0.09.  Communications  type  was  not  significant,  F(2,24)=l  .89,  p<.  1 7,  }jp  =0. 14.  Target  type 
was  significant,  suggesting  an  effect  of  the  size  of  targets  on  the  number  of  communications, 
/7(2,24)=  1 0.99,  /;<.()()  1 ,  rjp  =  48.  Using  raw  frequencies,  the  mean  count  for  the  large  target  was 
M=  5.88,  for  the  small  target  was  M=  7.48,  and  for  the  medium  target  was  M=  7.73.  Post-hoc  tests 
using  the  Bonferroni  adjustment  for  multiple  comparisons  revealed  that  Soldiers  did  require 
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significantly  fewer  communications  to  identify  the  large  target  versus  both  the  small  and  medium 
targets,  but  the  small  target  results  did  not  differ  from  the  medium  target  results.  The  significant 
post-hoc  results  are  shown  in  table  3  using  the  log  transformed  data. 

Table  3.  Communication  count  pos-  hoc  tests  on  target  type. 


Mean  Diff 

SE 

Sig 

d' 

Large  target 

Medium  target 

0.30 

0.07 

0.002 

1.35 

Large  target 

Small  target 

0.27 

0.06 

0.003 

0.93 

The  log  linear  analysis  supported  the  results  of  the  previous  ANOVA.  Starting  with  the  saturated 
model,  the  final  model  kicked  out  the  interaction  tenn  leaving  only  the  first  order  effects.  Of  the 
two  first  order  effects,  target  type  clearly  had  the  stronger  impact  on  the  model  with  a  significant 
log  likelihood  chi  square  value  nearly  3x  as  large  as  the  likelihood  value  for  communications 
type.  The  final  model  with  both  first  order  effects  was  not  significant  (using  a  strict  alpha 
criterion  of  .05)  x~(4,1648)=9.08,  p<.059.  However,  the  models  with  either  target  type  or 
communications  type  alone  were  significant.  The  log  likelihood  chi  square  value 
%2(2, 1 648)=8.96,  p<.0 1  1  for  communications  type  reveals  that  fewer  communications  were 
observed  in  the  tactor  condition  relative  to  the  chat  and  radio  conditions  than  were  expected. 

The  observed  and  expected  frequencies  are  shown  in  table  4.  Likewise,  'i  (2,1648)=22.09, 
p<0.001  for  target  type  indicates  that  fewer  communications  were  observed  for  the  large  target 
condition  relative  to  the  small  and  medium  conditions  than  were  expected. 


Table  4.  Communication  frequencies. 


Observed  N 

Expected  N 

Residual 

Tactor 

498 

549.3 

-51.3 

Chat 

597 

549.3 

47.7 

Radio 

553 

549.3 

3.7 

Total 

1648 

— 

— 

3.3  Exploratory  Analysis 
3.3.1  Data  Screening  and  Selection 

The  subjective  threshold  that  was  noted  regarding  the  150-s  ceiling  for  errors  raised  some 
questions  about  the  data  in  hindsight.  Research  on  human  error  (Horrey  and  Wickens,  2007) 
suggests  that  often  the  most  important  details  regarding  error  can  be  in  the  tails  of  the 
distribution,  and  that,  studies  focusing  on  the  centrality  of  the  distributions  may  mask  important 
evidence.  Consequently,  the  upper  and  lower  halves  of  the  time-to-identify-targets  and 
communication-frequency  data  sets  were  contrasted  for  differences.  The  median  of  each  data  set 
was  computed  and  the  data  recoded  as  either  above  or  below  the  median. 


12 


3.3.2  Analyses 

Chi  square  analyses  were  run  on  the  median  split  data  for  time  and  communication  frequency. 

3.3.3  Results 

The  results  of  the  chi  square  on  the  median  split  time  data  did  not  show  evidence  of  a 
proportional  difference  in  time  to  identify  targets  across  communication  types.  The  pattern  of 
instances  shown  in  table  5  reflects  a  greater  proportion  of  trials  below  the  median  than  above  it 
in  the  tactor  condition  relative  to  the  chat  and  radio  conditions.  This  pattern  was  not  inferentially 
supported  though  given  %2( 2 ,  1 17)=0.27,  p<0.87,  Cramer’s  V=0.05. 


Table  5.  Time  to  identify  targets:  median  split. 


Tactor 

Chat 

Radio 

Total 

Below  median 

21 

19 

19 

59 

Above  median 

18 

20 

20 

58 

Total 

39 

39 

39 

117 

The  results  of  the  median  split  on  communication  frequency  did  show  evidence  of  a  proportional 
difference  in  frequency  of  communications  across  communication  type.  The  pattern  of  instances 
shown  in  table  6  reflects  a  greater  proportion  of  trials  below  the  median  than  above  it  in  the 
tactor  condition  relative  to  the  chat  and  radio  conditions.  The  y  test  did  not  show  statistical 
significance  at  the  conventional  a=  0.05  level  but  this  pattern  did  show  evidence  of  inferential 
support  given  %2(2,  1 17)=5.41,  p<0.067,  Cramer’s  V=0.22. 


Table  6.  Communication  frequency  median  split. 


Tactor 

Chat 

Radio 

Total 

Below  median 

26 

16 

19 

61 

Above  median 

13 

23 

20 

56 

Total 

39 

39 

39 

117 

3.4  NASA  TLX  and  SART  Analyses 

3.4.1  Data  Screening  and  Selection 

Both  the  NASA  TLX  and  SART  data  sets  were  screened  for  illegal  values.  None  were  found. 
The  rating  data  revealed  non-normality  using  the  Shapiro-Wilk  test  of  normality  and 
consequently  treated  ordinally  in  the  analysis  to  resolve  the  lack  of  normality. 

3.4.2  Analyses 

Both  data  sets  were  analyzed  using  the  Friedman  nonparametric  test  for  dependent  samples. 
Only  the  primary  variable  of  interest,  communication  type,  was  used  in  this  analysis. 
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3.4.3  Results 

None  of  the  six  dimensions  of  workload  showed  any  evidence  of  differences  between 
communication  types.  Likewise,  none  of  the  three  dimensions  of  SA  indicated  any  differences 
between  communication  types. 


4.  Conclusion 


There  was  considerable  noise  in  the  data  caused  by  intermittent  problems  with  the  TALON, 
among  other  uncontrollable  factors.  The  power  analysis  allowed  us  to  conclude  that  additional 
participants  would  not  have  changed  the  results.  In  particular,  we  found  that  communication 
mode  has  no  real  effect  on  target  identification  time,  i.e.,  no  mode  performed  significantly  better 
or  worse  than  the  others.  This  in  itself  is  an  important  finding  because  it  suggests  that  the 
tactical  use  of  haptic  signals  for  covert  communications  may  be  feasible  in  light  of  finding  no 
evidence  to  support  differential  performance  within  the  haptic  form  of  communication.  Thus, 
haptics  could  potentially  be  a  desirable  tactical  option.  While  not  implying  performance  is 
equivalent  among  communication  modes,  we  obtained  no  evidence  that  would  lead  us  to  believe 
the  converse,  i.e.,  there  was  a  performance  decrement  for  tactile  communications.  This  is 
gratifying  because  of  the  difficulty  of  the  task,  which  consisted  of  finding  targets  in  a  previously 
unknown  environment  using  an  unfamiliar  tactile  lexicon  while  navigating  a  TALON  robot  with 
which  they  had  limited  familiarity. 

Gilson  et  al.  (2007)  suggested  that  robotic  operators  will  be  able  to  use  low  bandwidth  tactile 
signals  to  communicate  covertly  with  their  chain  of  command  and  eventually  communicate  with 
future  robots  with  enhanced  cognitive  abilities.  Perhaps  of  equal  importance  is  that  tactile 
communications  permit  Soldiers  to  use  a  virtually  unused  communications  channel  to  increase 
SA  without  impinging  on  their  visual  or  auditory  resources  (Bames,  in  press).  Also,  there  was 
evidence  that  the  tactile  cueing  might  actually  be  an  efficient  means  of  communication,  as 
reflected  in  the  results  that  indicted  a  reduced  number  of  communications  were  necessary  for 
target  location  in  this  mode;  however,  this  finding  requires  additional  research  for  verification. 

Also,  it  is  important  to  note  that  the  13  Soldiers  were  relatively  efficient  in  using  the  TALON  to 
find  targets  in  desert  terrain.  It  would  be  a  mistake  to  assume  this  was  an  easy  task.  The  targets 
were  hidden  in  the  desert  foliage  so  that  the  targets  were  never  in  plain  sight  of  the  robot  sensor 
system.  Also,  during  this  time  of  year,  there  was  a  great  deal  of  vegetation  on  the  desert  floor 
including  complex  sand  dunes  overgrown  with  thorny  creosote  bushes  because  of  the  monsoon 
rains.  The  training  was  minimal;  total  time  was  >1  h.  The  training  took  place  in  a  separate  area 
to  ensure  that  Soldiers  were  not  familiar  with  target  locations.  Nevertheless,  the  Soldiers  were 
able  to  navigate  the  TALON  to  most  of  the  targets  rapidly.  There  was  a  consistent  target  effect. 
Logically,  it  would  seem  the  small  target  would  be  the  most  difficult  target  to  find  and 
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consequently  reflect  a  great  search  time  and/or  communication  frequency.  This  was  not  the  case. 
Rather,  the  results  showed  evidence  that  medium  targets  were  the  most  challenging  to  identify. 

In  summary,  the  results  reinforced  evidence  from  previous  research  that  tactile  communications 
are  a  tactically  useful  fonn  of  communication  for  Soldier-robot  interactions. 

Past  research  has  shown  that  the  tactile  channel  is  beneficial  when  used  redundantly  with  either 
the  auditory  or  visual  channel.  In  these  paired  instances,  a  tactile  cue  may  increase  both  speed 
and  accuracy  of  Soldier  responses  (e.g.,  Hempel  and  Altinsoy,  2005).  The  current  research 
suggests  that  in  environmental  manipulations  that  make  nonnal  communications  modalities 
ineffective,  the  tactile  channel  may  serve  as  a  tactically  effective  alternative.  The  findings  in  this 
experiment  suggest  that  the  inclusion  of  the  tactile  mode  for  communication  poses  no  detriment 
in  perfonnance,  and  perhaps  even  an  enhancement.  These  results  make  a  good  argument  to 
examine  transitioning  this  communications  channel  to  a  variety  of  operations.  Subsequent 
experimentation  should  include  examining  chat,  radio,  and  tactile  communication  modes  in  day 
versus  night  or  safe  versus  covert  environments,  and  possibly  further  exploring  tactile 
communications  as  a  redundant  channel  within  the  various  environments. 
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